High dimensional predictions of suicide risk in 4.2 million US Veterans using ensemble transfer learning

We present an ensemble transfer learning method to predict suicide from Veterans Affairs (VA) electronic medical records (EMR). A diverse set of base models was trained to predict a binary outcome constructed from reported suicide, suicide attempt, and overdose diagnoses with varying choices of study design and prediction methodology. Each model used twenty cross-sectional and 190 longitudinal variables observed in eight time intervals covering 7.5 years prior to the time of prediction. Ensembles of seven base models were created and fine-tuned with ten variables expected to change with study design and outcome definition in order to predict suicide and combined outcome in a prospective cohort. The ensemble models achieved c-statistics of 0.73 on 2-year suicide risk and 0.83 on the combined outcome when predicting on a prospective cohort of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sim$$\end{document}∼ 4.2 M veterans. The ensembles rely on nonlinear base models trained using a matched retrospective nested case-control (Rcc) study cohort and show good calibration across a diversity of subgroups, including risk strata, age, sex, race, and level of healthcare utilization. In addition, a linear Rcc base model provided a rich set of biological predictors, including indicators of suicide, substance use disorder, mental health diagnoses and treatments, hypoxia and vascular damage, and demographics.


Supplementary Discussion
Several indicators of suicide, including diagnosis of suicide attempts and suicidal ideation, are among the strongest predictors of suicide and our combined outcome.PHQ2 and PHQ9 scores 1 displayed clear associations, with increasing risk from higher scores.A similar finding was noted by Simon et al. 2 .While use of overlapping coefficients provides more information to the predictive model, it complicates interpretation and, as evidenced by the subgroup analysis in Supplementary Figure S10, requires non-linear models to accurately account for the overlaps among categories.It is also important in this respect that all of our variables were encoded as 0/1 variables, with NA included as a separate category and normal ranges used as reference.Univariate analyses, such as used in Supplementary Table S3 and the calibration plots in the risk histograms at the bottom of Supplementary Figures S9 -S12 are also valuable to understand overlap of categories.
Mental health diagnoses showed plausible trends, with bipolar and personality disorders most strongly associated with our combined outcome.Behavioral health treatments are the strongest predictors for suicide, but there is likely a selection bias as they are potentially offered to patients at the most significant risk of negative outcomes.The difficulties in controlling for this bias is similar to the problem of comparative effectiveness analysis discussed at length in, for example, reference 3 .
Among behavioral health medications, barbiturates, antidepressants, anti-anxiety medications, and opioids for pain are associated with our combined outcome, while stimulants and opioid antagonists such as naloxone were protective.A study conducted in the Netherlands found an increase in suicides resulting from barbiturate overdose 4 .In 2004, the FDA issued a black box warning on SSRI antidepressants as they increase the risk of suicide.It has been suggested that certain depressed patients might already be harboring suicidal thoughts, but would not attempt suicide until their energy is increased as a result of the antidepressant treatment 5 .Medication classes like SSRI antidepressants and benzodiazepines serve as anti-anxiety medications.Benzodiazepines may increase suicide risk in patients by increasing impulsivity or aggression, causing rebound or withdrawal symptoms, and toxicity in overdose 6 .A cohort study of patients with ADHD found a statistically significant protective effect of stimulants such as methylphenidate on suicidal behavior 7 , consistant with the protective effect indicated for stimulants in Figure 4.It is proposed that their protective effect may be mediated by the improvement of ADHD symptoms, especially impulsivity 7 .
Opioid users have been reported to be fourteen times more likely to die by suicide than the general population 8 .Chronic opioid use alters the limbic and striatal brain circuitry, escalating negative affect and contributing to the mental pain that accompanies suicidal behavior 9 .Our results show opioid overdose and opioid use disorder (OUD) are positively associated with our combined outcomes, while Figure 6b shows the association of opioid related variables increases suicide risk more than suicide attempts.A study conducted in the United States reported that drug overdoses from opioids accounted for roughly 40% of fatal suicide poisonings 10 .Patients with OUD have increased amygdala activation and this activation enhances negative emotional reactivity in healthy adults 11 and suicidal ideation in depressed patients.Opioid overdoses among metastatic cancer patients were examined in 12 , who found no elevation in OUD but did see that overdoses more frequently occurred in advanced cancers, among patients with pre-existing mental health diagnoses, and those without previous difficulties with opioid use.Naloxone is an opioid antagonist medication that is used to reverse an opioid overdose.Figure 6a shows a striking shift in the ratio of suicide attempts and overdose, depending on whether the patient's age is above or below sixty years of age.
A 2018 study of 7,418,391 individuals living in Denmark reported that patients with severe traumatic brain injury (TBI) had a 2.38 adjusted increased rate ratio (IRR) of suicide compared to the general population, with a 1.81 IRR for mild TBI 13 .
Figure 4 shows the largest risk of suicide with Dx codes for TBI, with concussion showing only a small model coefficient.When comparing to literature, it is important to realize that our model coefficients are adjusted for indicators of suicide, while literature values are typically not.The authors of the Danish study proposed the association between TBI and suicide was at least partially mediated by post-TBI psychiatric symptoms 13 .
Elevated pulse rate and pulse pressure are associated with our combined outcome, consistent with the prospective study by Lemogne 14 , as are decreased blood oxygenation and anemia.Our work confirms prior literature suggesting has that markers of chronic hypoxia such as smoking, living at high altitude and diagnosis of chronic obstructive pulmonary disease (COPD) or asthma may lead to reduced brain serotonin synthesis, depressing mood and contributing to suicide 15 .A study of the association of elevation to suicide risk identified a hypoxia link, although other variables such as gun ownership and reduced access to medical care must be disentangled from this association 16 .
Coronary artery disease (CAD), electrolyte imbalances, and cardiac arrhythmias are predictive of our combined outcome, but heart attacks and diabetes were not.Evidence suggests multiple mechanisms by which anger or hostility may be linked to CAD 17 .It has been hypothesized that this association can also be explained by shared common genetic factors associated with vascular vulnerability which appears to be modulated by high risk environments 18 .Cardiac arrhythmia is most commonly caused by electrolyte imbalance.Low intracellular concentration of electrolytes such as potassium has been observed among depressed patients and also found in the cerebral tissue of patients who died by suicide 19 .
We included a dozen variables from the American Community Survey 20 in our analysis, including variables characterizing poverty, disability, insurance, and other demographics.Only two of these variables showed effects above the 20% level: zip codes with relatively high fractions with Native American race and zip codes with a relatively high fraction of Spanish-speaking population.The decreased risk associated with Native Americans is perhaps surprising, given the relatively high suicide rates associated with such communities 21 , suggesting that a less-sensitive reporting of events may be the relevant attribute.
Marital status change, including divorce, death of spouse, and new marriages are predictive of the combined outcomes.Being married in contrast, is protective in comparison with not being married.A Norwegian study concluded that being divorced or separated is associated with higher suicide risk than never being married, which they attributed to weakened social ties 22 .
Our results, represented in Figure 4 shows obesity is protective against the combined outcomes, consistent with reduced levels of suicide ideation reported in studies in 2009 23 , of 33,249 individuals drawn from the South Austrailian population and 2022 24 , of 13,943 young US men and women chosen to be representative of the US population.S1.C-statistics for outcome prediction from models trained on the seven methodologies (rows) applied to predict outcomes on the Rcc and C17 cohorts.For each combination, the top score is AUROC for the model used directly on the evaluation set.The bottom number (fine-tuned) is obtained by re-fitting a logistic regression model with the score from the previous model along with demographics gender, race, ethnicity, and age, and all eight time bins for four variables: Pain ¿ 0, suicidal ideation, suicide attempt, and overdose in a logistic regression, together with original model scores.For the nonlinear models, log(score/(1-score)) was included in the fine-tuning rather than the model score directly.For models where differences between test and train accuracies, both are provided in the order train/test cohorts.Additionally, for the logistic regression and Cox models, the 'DoF' column provides the number of degrees of freedom that survived LASSO model selection.* For the C17 cohort, controls were randomly down-selected by a factor of four in each date-of-birth cohort.Numbers in this table reflect down selected groups, while numbers in Figure 1 reflect quantities before down selection.Table S3.Attributes of predictor variables associated with combined outcome for Rcc study design.Selected coefficients are shown in the first column, for which the following information is provided.Model selected logistic regression coefficients are provided for the C17, Cox-MH, and Rcc Cohorts, together with the adjusted odds ratio for Rcc (exp(Coe f .Rcc)).Next are provided the number of cases and controls with each attribute, and the percentage of patients with each attribute that are cases.Note that the total number of casess and controls are provided for reference in the top row.This is followed by the unadjusted hazard ratio (%cases/%controls for each variable) for Rcc and the relative risks for each component of our combined outcome, also for Rcc cohort.

Figure S8 . 11 Figure S9 .
Figure S8.Model coefficients for C17 cohort related to non mental health related predictors

Figure S10 . 11 Figure S11 .
Figure S10.Calibration curves (top) and histograms of scores (bottom) for all patients (black) and four subgroups of patients are defined by the presence of selected variables observed to be strongly predictive of our combined outcome.Curves for patients with high PHQ2 scores are in red, patients visiting the Emergency Department are in green, patients receiving a diagnosis code indicating injury from a fall are in blue, and patients receiving a diagnosis code of suicidal ideation are in cyan.The two left sets of panels show curves for our ensemble models for combined outcome and suicide outcome.The two right sets of panels show curves for our C15 logistic regression model, fine-tuned and evaluated against the C17 cohort.

Figure S12 .
Figure S12.Calibration curves (top) and histograms of scores (bottom) for all male patients (black) and five subgroups of patients, defined by the gender (female) in red, race (African American) in green, ethnicity (Hispanic) in blue, age less than 50 years in cyan and greater than 65 in magenta.The two left sets of panels show curves for our ensemble models for combined outcome and suicide outcome.The two right sets of panels show curves for our C15 logistic regression model, fine-tuned and evaluated against the C17 cohort.

Table S2 .
Characteristics of the study population.Study size and number of cases are provided for retrospective case-control (Rcc), time-to-event (Cox) models for Cox-MH and Cox-Visit study designs, and interval predictions for C17.Number of cases and controls are provided for all four studies, and the subset of Cases with an NDI-reported suicide is provided for Rcc and Calendar-2017.Cox-Visit and C17 were both nested cohort studies, downsampling controls by a factor of four, indiciated by the SubCohort row.